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I. 



Statement of Real Party in Interest 



The real party in interest in the above-identified patent application is the International 
Business Machines Corporation. 

II. Statement of Related Proceedings 

There are no prior or pending appeals or interferences related to this application to 
Appellant's knowledge. 

III. Statement of Supporting Evidence 

Applicants are not relying on any affidavits, extrinsic documents or extrinsic 
evidence. 

IV. Statement of Claim Status and Appealed Claims 
A. Claim Status 

Claim 1 stands rejected under 35 U.S.C. §103 as being unpatentable over U.S. Patent 
No. 5,767,842 (Korth) in view of U.S. Patent 6,407,679 (Evans, et al.), and under 35 
U.S.C. §112, first paragraph, as failing to comply with the written description 
requirement. 

Claim 2 stands rejected under 35 U.S.C. §103 as being unpatentable over U.S. Patent 
No. 5,767,842 (Korth) in view of U.S. Patent 6,407,679 (Evans, et al.), and under 35 
U.S.C. §112, first paragraph, as failing to comply with the written description 
requirement. 

Claim 3 stands rejected under 35 U.S.C. §103 as being unpatentable over U.S. Patent 
No. 5,767,842 (Korth) in view of U.S. Patent 6,407,679 (Evans, et al.), and under 35 
U.S.C. §112, first paragraph, as failing to comply with the written description 
requirement. 

Claim 4 stands rejected under 35 U.S.C. §103 as being unpatentable over U.S. Patent 
No. 5,767,842 (Korth) in view of U.S. Patent 6,407,679 (Evans, et al.), and under 35 
U.S.C. §112, first paragraph, as failing to comply with the written description 
requirement. 

Claim 5 stands rejected under 35 U.S.C. §103 as being unpatentable over U.S. Patent 
No. 5,767,842 (Korth) in view of U.S. Patent 6,407,679 (Evans, et al.), and under 35 
U.S.C. §112, first paragraph, as failing to comply with the written description 
requirement. 

Claim 6 stands rejected under 35 U.S.C. §103 as being unpatentable over U.S. Patent 
No. 5,767,842 (Korth) in view of U.S. Patent 6,407,679 (Evans, et al.), and under 35 
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U.S.C. §112, first paragraph, as failing to comply with the written description 
requirement. 

Claim 7 stands rejected under 35 U.S.C. §103 as being unpatentable over U.S. Patent 
No. 5,767,842 (Korth) in view of U.S. Patent 6,407,679 (Evans, et al.) 5 and under 35 
U.S.C. §112, first paragraph, as failing to comply with the written description 
requirement. 

Claim 8 stands rejected under 35 U.S.C. §103 as being unpatentable over U.S. Patent 
No. 5,767,842 (Korth) in view of U.S. Patent 6,407,679 (Evans, et al.), and under 35 
U.S.C. §112, first paragraph, as failing to comply with the written requirement. 

Claim 9 stands rejected under 35 U.S.C. §103 as being unpatentable over U.S. Patent 
No. 5,767,842 (Korth) in view of U.S. Patent 6,407,679 (Evans, et al), and under 35 
U.S.C. §112, first paragraph, as failing to comply with the written description 
requirement. 

Claim 10 stands rejected under 35 U.S.C. §103 as being unpatentable over U.S. 
Patent No. 5,767,842 (Korth) in view of U.S. Patent 6,407,679 (Evans, et al.), and 
under 35 U.S.C. §112, first paragraph, as failing to comply with the written 
description requirement. 

Claim 11 stands rejected under 35 U.S.C. §103 as being unpatentable over U.S. 
Patent No. 5,767,842 (Korth) in view of U.S. Patent 6,407,679 (Evans, et al.), and 
under 35 U.S.C. §112, first paragraph, as failing to comply with the written 
description requirement. 

Claim 12 stands rejected under 35 U.S.C. §103 as being unpatentable over U.S. 
Patent No. 5,767,842 (Korth) in view of U.S. Patent 6,407,679 (Evans, et al.), and 
under 35 U.S.C. §112, first paragraph, as failing to comply with the written 
description requirement. 

Claim 14 stands rejected under 35 U.S.C. §103 as being unpatentable over U.S. 
Patent No. 5,767,842 (Korth) in view of U.S. Patent 6,407,679 (Evans, et al.), and 
under 35 U.S.C. §112, first paragraph, as failing to comply with the written 
description requirement. 

Claim 15 stands rejected under 35 U.S.C. §103 as being unpatentable over U.S. 
Patent No. 5,767,842 (Korth) in view of U.S. Patent 6,407,679 (Evans, et al.), and 
under 35 U.S.C. §112, first paragraph, as failing to comply with the written 
description requirement. 

Claim 16 stands rejected under 35 U.S.C. §103 as being unpatentable over U.S. 
Patent No. 5,767,842 (Korth) in view of U.S. Patent 6,407,679 (Evans, et al.), and 
under 35 U.S.C. §112, first paragraph, as failing to comply with the written 
description requirement. 
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Claim 17 stands rejected under 35 U.S.C. §103 as being unpatentable over U.S. 
Patent No. 5,767,842 (Korth) in view of U.S. Patent 6,407,679 (Evans, et al.), and 
under 35 U.S.C. §112, first paragraph, as failing to comply with the written 
description requirement. 

Claim 18 stands rejected under 35 U.S.C. §103 as being unpatentable over U.S. 
Patent No. 5,767,842 (Korth) in view of U.S. Patent 6,407,679 (Evans, et al.), and 
under 35 U.S.C. §112, first paragraph, as failing to comply with the written 
description requirement. 

Claim 19 stands rejected under 35 U.S.C. §103 as being unpatentable over U.S. 
Patent No. 5,767,842 (Korth) in view of U.S. Patent 6,407,679 (Evans, et al), and 
under 35 U.S.C. §112, first paragraph, as failing to comply with the written 
description requirement. 

Claim 21 stands rejected under 35 U.S.C. §103 as being unpatentable over U.S. 
Patent No. 5,767,842 (Korth) in view of U.S. Patent 6,407,679 (Evans, et al.), and 
under 35 U.S.C. §112, first paragraph, as failing to comply with the written 
description requirement. 

Claim 22 stands rejected under 35 U.S.C. §103 as being unpatentable over U.S. 
Patent No. 5,767,842 (Korth) in view of U.S. Patent 6,407,679 (Evans, et al), and 
under 35 U.S.C. §112, first paragraph, as failing to comply with the written 
description requirement. 

Claim 23 stands rejected under 35 U.S.C. §103 as being unpatentable over U.S. 
Patent No. 5,767,842 (Korth) in view of U.S. Patent 6,407,679 (Evans, et al.), and 
under 35 U.S.C. §112, first paragraph, as failing to comply with the written 
description requirement. 

Claim 24 stands rejected under 35 U.S.C. §103 as being unpatentable over U.S. 
Patent No. 5,767,842 (Korth) in view of U.S. Patent 6,407,679 (Evans, et al.), and 
under 35 U.S.C. §112, first paragraph, as failing to comply with the written 
description requirement. 



B. Appealed Claims 

Claims 1-12, 14-19 and 21-24 are appealed. A clean copy of these claims is 
contained in Appendix A to this Appeal Brief. 
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V. Statement of Amendment Status 

No amendments are pending in this application. The last Amendment filed in this 
case was dated January 3, 2005. That Amendment has been entered. 

VI. Statement/Explanation of Invention 

The present invention, generally, relates to methods and systems for generating text 
or data from typing gestures made without any real physical keyboard. In the practice 
of this invention, a person (103) moves his or her hands as if that person was typing, 
and using various computer processes (200, 201, 202, 203), those finger movements 
are transformed into text (204). In particular, these computer processes (200, 201, 
202, 203) classify the finger gestures into classes, and then associated each of those 
classes with one of the keys of a keyboard. 

With preferred embodiment of the invention, images of the finger gestures (500) are 
taken. The computer processes are then used to classify these gesture images into 
classes (503) to associate each of those classes with one or more possible keys (504) 
and to assign a probability to each of those possible keys. After a sequence of 
classes, the probabilities assigned to the possible keys are integrated to identify a 
word for that sequence of classes. Each word is identified on the basis of the 
probabilities assigned to the possible keys assigned to the gesture classes that make 
up that word. 

VII. Statement/List of Each Ground for Review 

1 . Rejection under 35 U.S.C. 1 12, first paragraph. 

a. The rejection 

Claims 1-12, 14-19 and 21-24 are rejected under 35 U.S.C. 112, first paragraph, 
as failing to comply with the written description requirement. 

b. Argument 

With respect to the rejection of the claims under 35 U.S.C. §1 12, the Examiner argued in 
the Office Action dated November 1, 2004, that the specification does not provide the 
appropriate written support for the description in the claims of the features that the typing 
gestures are "made in a touch-sensor free environment." Applicants, in the Amendment 
dated January 3, 2005 and which was entered, slightly rephrased this language in the 
claims to indicate, more specifically, that the typing gestures are made "without touching 
any touch sensors." In order to provide an express support for this language in the 
claims, the specification on page 5 was also amended in that amendment to indicate that 
the typing gestures shown in Figure 1 are made without touching any touch sensors. 
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This feature is clearly shown in Figure 1 of the application, and is readily understood by 
those of ordinary skill in the art from a review of that Figure and the discussion thereof 
on pages 4 and 5 of the application. Moreover, with these changes, the application now 
provides the appropriate written description of the claimed subject matter. 

In view of the foregoing, the Examiner's rejection of the claims under 35 U.S.C. §112 has 
been overcome, and the Board is asked to reverse the rejection of claims 1-12, 14-19 and 
21-24 under 35 U.S.C. §112. 

2. Rejection under 35 U.S.C. 103 based on U.S. Patents 5,767,842 (Korth) and 
6,407,679 (Evans, et al.). 

a. The rejection 

Claims 1-12, 14-19 and 21-24 are rejected under 35 U.S.C. 103 as being unpatentable 
over U.S. Patent 5,767,842 (Korth) in view of U.S. Patent No. 6,407,679 (Evans, et al.). 

b. U.S. Patent 5,767,842 

Korth describes a virtual typing procedure in which hand and finger movements are 
interpreted as operations on a physically non-existent computer keyboard or similar input 
device. In this procedure, an image acquisition system is provided to monitor the hand 
and finger motions and gestures of a human use, and these activities are interpreted as 
operations on a keyboard or other input device. This image acquisition system comprises 
a video sensor, and means for digitizing the processing signals from that sensor. 

The Examiner has noted that several important aspects of the invention are not disclosed 
in Korth, and thus, the Examiner relies on Evans, et al. to support the rejection of the 
claims. 

c. U.S. Patent No. 6,407, 679 

Evans, et al discloses a procedure for converting finger movement into text. With the 
procedure disclosed in Evans, et al, each finger gesture is assigned a number, a series of 
numbers is formed, and that series is matched to one or more possible words. After a 
sentence is completed, all possible sentences are formed, a probability is assigned to each 
sentence possibility, and the most probable sentence is selected. 

d. Features of the claims that are not disclosed or suggested by the prior art. 

The following features of the present inventions are not disclosed in or suggested by the 
prior art. 

(i) A virtual typing procedure, as described in independent claims 1, 
4, 8, 9, 10, 14, 17 and 21, that uses a virtual and invisible 
keyboard, as described in these claims. 
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(ii) A virtual typing procedure, as described in independent claims 1, 
4, 8, 9, 10, 14, 17 and 21, in which each word is identified on the 
basis of the probabilities assigned to the gesture classes that make 
up that word . 

e. Argument 

As indicated above, neither Korth nor Evans, et al. discloses or suggests a virtual typing 
procedure that uses a virtual and invisible keyboard. More specifically, Korth assumes 
that the user sees a virtual keyboard, either printed on a table or seen on a monitor. With 
the present invention, in contrast, the user does not see a virtual keyboard at all. The user 
may not even have a table in front of him. Because of this, in this invention, the typing 
processes are a set of gestures that imitate some typical typing movements but they are so 
vague that they require a statistical recognition process that interprets gestures, similar to 
the way a speech recognition machine interprets speech. 

It is noted that, in the Office Action of November 1, 2004 the Examiner argued that Korth 
discloses a physically non-existent input device and that this corresponds to an invisible 
virtual keyboard. Applicants respectfully submit that a virtual keyboard is not the same 
as a virtual invisible keyboard. The virtual keyboard of Korth is clearly visible; this is 
plainly shown in Figure 1 of Korth. In the present invention, in contrast, the keyboard is 
not only virtual (there is no real keyboard), but it also is not visible (there is no image of 
any keyboard, and it cannot be seen). The keyboard exists only in the mind of the user. 
Further, it is the use of this invisible keyboard that necessitates the above-discussed 
gesture analysis procedure that is needed in the operation of the present invention. 

Each of independent claims 1, 4, 8, 9, 10, 14, 17 and 21 describes a virtual invisible 
keyboard. More specifically, Claim 1 describes an information input processing 
computer system for mapping gestures to keys of an invisible keyboard, and claim 4 
describes an information processing, gesture-key mapping computer system for 
generating text from hand gestures of a user relative to an invisible keyboard. Claims 8 
and 9 both describe a method for producing a textual output in which a user makes typing 
like gestures relative to a virtual, invisible keyboard. Claims 10 and 14 are directed to a 
method of typing, and both of these claims positively set forth the step of making typing 
gestures relative to a virtual, invisible keyboard. Analogously, Claims 17 and 21 are 
directed to a typing system using a virtual invisible keyboard, and these claims set forth 
means for sensing typing gestures made relative to a virtual, invisible keyboard. 

In addition to the foregoing, another important feature of this invention that is not shown 
in or suggested by the prior art, is the specific way in which the present invention 
translates gestures into text. 

As mentioned above, with the present invention, images of the gestures are taken or 
captured, and the computer processes are used to classify gesture images into classes, to 
associate each of those classes with one or more possible keys, and to assign a probability 
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to each of those possible keys. Then, for a sequence of classes, the probabilities assigned 
to the possible keys associated with the sequence are integrated to identify a word for that 
sequence of classes. 

Korth discloses a virtual typing system in which hand and finger movements are 
interpreted as operations on a physically non-existent computer keyboard or other input 
device. The Examiner has noted in previous Office Actions (see Office Action of July 
16, 2003, page 3, lines 5-7) that Korth does not disclose classifying gestures into classes 
and associating each of those classes with one of the keys of the keyboard. The 
Examiner argued, though, that Evans, et al teaches these or similar functions. 

Evans, et al discloses a procedure for converting finger movement into text. However, 
the specific procedure used in Evans, et al to perform that conversion is quite different 
than the specific procedure used in the practice of the present invention. 

To elaborate, with the procedure disclosed in Evans, et al, each finger gesture is assigned 
a number, a series of numbers is formed, and that series is matched to one or more 
possible words. After a sentence is completed, all possible sentences are formed, a 
probability is assigned to each sentence possibility, and the most probable sentence is 
selected. 

In contrast, with this invention, text is determined on a word by word basis rather than on 
a sentence by sentence basis. With the present invention, each word is identified on the 
basis of the probabilities assigned to the possible keys assigned to the gesture classes that 
make up that word. 

This feature is of utility because it provides a quicker feedback to the user. With the 
procedure used in Evans, et al, a complete sentence must be finished before a final text is 
determined. The present invention, in comparison, is able to determine each word 
immediately after the word is input. 

Independent Claims 1, 4, 8, 9, 10, 14, 17 and 21 clearly describe this feature of the 
invention. More specifically, each of Claims 1, 8, 9, 10 and 17 indicates that the 
computer processes described in the claims, among other functions, associate each of the 
classes with one or more possible keys, assign a probability to each of those possible 
keys, and integrate the probabilities assigned to the possible keys to identify a word for a 
sequence of gestures. 

Claim 4 indicates that the associator module associates gesture classes with one or more 
possible keys, and that the integrator module integrates the probabilities assigned to the 
possible keys to identify a word for a sequence of gestures. In addition, Claims 14 
describes the producing step as including the step of associating each of the classes with 
one or more possible keys, assigning a probability to each of said possible keys, and 
integrating the probabilities assigned to the possible keys to identify a word for a 
sequence of gestures. Claim 21, which is directed to a typing system, includes analogous 
apparatus limitations. 
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Appellants respectfully submit that the important consideration is not whether the 
reference could be modified to produce the present invention, but whether it would have 
been obvious to do so. 

As the Court of Appeals for the Federal Circuit has taught: 

"an 'obvious to try 5 standard is not a legitimate test of patentability... The statutory 
standard of 103 is whether the invention, considered as a whole, would have been 
obvious to one skilled in the art, not whether it would have been obvious to one skilled in 
the art to try various combinations." N.V. Akzo v. E.I. duPont deNemours & Co. , 1 
USPQ2d 1704, 1707 (Fed Cir. 1987). 

Appellants submit that, because of the above-discussed differences between the prior art 
and Claims 1, 4, 8, 9, 10, 14, 17 and 21, these claims, when judged according to the 
proper standard, patentably distinguish over the prior art. Claims 2 and 3 are dependent 
from Claim 1 and are allowable therewith; Claims 5, 6 and 7 are dependent from, and are 
allowable with, Claim 4; and Claims 11, 12, 22 and 23 are dependent from Claim 10 and 
are allowable therewith. Similarly, Claims 15 and 16 are dependent from, and are 
allowable with, Claim 14; Claims 18 and 19 are dependent from Claim 17 and are 
allowable therewith; and Claim 22 is dependent from, and is allowable with, Claim 21. 



VIII. Conclusion 

The specification provides the appropriate written support, within the meaning of 35 
U.S.C. §112, first paragraph, for claims 1-12, 14-19 and 21-24. In particular, the 
specification includes a written description of the feature that the typing gestures are 
made "without touching any touch sensor." Accordingly, the Board of Appeals is 
requested to reverse the rejection of claims 1-12, 14-19, and 21-24 under 35 U.S.C. §112. 

The prior art, and specifically, Korth and Evans, et al. does not disclose or suggest a 
number of important features of the present invention that are described in independent 
Claims 1, 4, 8, 9, 10, 14, 17 and 21. Specifically, the prior art does not teach a virtual 
typing procedure, as described in these independent claims, that uses a virtual and 
invisible keyboard. Also, the prior art does not disclose or suggest a virtual typing 
procedure, as described in independent claims 1, 4, 8, 9, 10, 14, 17 and 21, in which each 
word is identified on the basis of the probabilities assigned to the gesture classes that 
make up that word. Accordingly, the rejection of these claims under 35 U.S.C. §103 is 
not proper. As indicated immediately above, the remaining pending claims as dependent 
from one of claims 1, 4, 8, 9, 10, 14, 17 or 21, and are thus also allowable. 
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The Board of Appeals is thus also respectfully asked to reverse the rejection of claims 
1-12, 14-19 and 21-24 under 35 U.S.C. §103. 



Scully, Scott, Murphy & Presser 
400 Garden City Plaza - Suite 300 
Garden City, New York 1 1530 
(516) 742-4343 

JSS:ej 

Enclosure: Appendix A 



Respectfully submitted, 



Dated: May 2, 2005 




Registration No. 28,757 
Attorney for Applicant 



10 



G:\Ibm\105\13317\amend\13317.Brief.doc 



APPENDIX "A" 

Claim 1 : An information input processing computer system for mapping gestures to keys of a 
virtual invisible keyboard, the system comprising one or several cameras, one or more memories 
with CPU connected to the cameras, and processes running in the CPU that associate gesture 
movements made without touching any touch sensors with typing and produce gesture associated 
textual output, wherein said processes capture gesture images, classify each type of gesture 
image into a respective one of a plurality of classes depending on the type of gesture, and 
associate each of the classes with one or more possible keys of the invisible keyboard, assign a 
probability to each of said possible keys, and integrate the probabilities assigned to the possible 
keys to identify a word for a sequence of gestures. 

Claim 2: The gesture-key mapping system as in claim 1, where a feedback is provided to the 
user on what kind of keys are associated with the user's gestures. 

Claim 3: The gesture-key mapping system as in claim 2, where the feedback is provided using 
one or more of the following: displaying keys on a display, playing sounds labels for keys, 
displaying image indicators on a display, playing special sound indicators, projecting the 
keyboard to any surface, and displaying picture of the keyboard with user's hands. 

Claim 4: An information input processing, gesture-key mapping computer system for generating 
text from hand gestures of a user relative to an invisible keyboard, the system comprising one or 
several cameras, one or more memories with CPU connected to the cameras, and processes 
running in the CPU that associate gesture movements with typing and produce gesture associated 
textual output, where the gesture-key processing is provided using the following modules: 

a) a gesture capturing module that captures gestures, relative to an invisible 
keyboard and made without touching any touch sensors, through camera 
sensors; 

b) a gesture classificator module that classifies each type of gesture into a 
respective one of a plurality of classes depending on the type of gesture 
movements; 
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c) an associator module for associating gesture classes or sequence of gesture 
classes with one or more possible keys of the invisible keyboard and assigns a 
probability to each of said possible keys; and 

d) an integrator module that integrates the probabilities assigned to the possible 
keys to identify a word for a sequence of gestures. 

Claim 5: The system as in claim 4, where the integrator module includes one or more of the 
following: 

a) language module component that estimate probabilities of word strings 
corresponding to key candidate sequences; 

b) character frequency module that estimate probabilities of character strings 
corresponding key candidate sequences; 

c) confusable matrix that estimate how often correct gesture classes are 
confusable with another gesture classes; 

d) gesture classes probability model that estimate probability of observing a string 
of gesture classes given a sequence of gesture frames; 

e) computation of a probability of production a sequence of keys given a string of 
gesture frames; 

f) generation of a lattice of sequences of keys given sequence of gesture frames; 

g) finding the most probable sequence of keys from the lattice of key candidate 
strings. 

Claim 6: A system according to Claim 5, wherein each sequence of keys receives a probability 
score, and the sequences of keys that receive low scores are removed and are not continuing 
when new candidates for keys arrive. 

Claim 7: A system according to Claim 4, further comprising a gesture correlator module to 
allow to adjust automatically the invisible keyboard to changes in the hand positions of the user, 
and wherein the gesture correlator module acts between the gesture classificator module and the 
associator module to maintain a consistent mapping of gesture classes to keys of the invisible 
keyboard despite said changes in the hand positions of the user. 
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Claim 8: The method for producing a textual output in which a user makes typing like gesture 
relative to a virtual, invisible keyboard made without touching any touch sensors and without the 
presence of a real keyboard and the gestures are associated with the most probable keys that 
would be typed if a keyboard were presented, said method including the steps of using a 
computer system to map gestures made, without touching any touch sensors, to keys of the 
virtual keyboard, including the steps of running processes on the computer to capture gesture 
images, to classify each type of gesture image into a respective one of a plurality of classes 
depending on the type of gesture, and to associate each of the classes with one or more possible 
keys of the invisible keyboard, assign a probability to each of said possible keys, and integrate 
the probabilities assigned to the possible keys to identify a word for a sequence of gestures. 

Claim 9: The method for producing a textual output in which a user makes typing like gestures 
relative to a virtual, invisible keyboard and without the presence of a real keyboard and the 
gestures are associated with the most probable keys that would be typed if a keyboard were 
presented, said method including the step of using a computer system to map gestures made, 
without touching any touch sensors, to keys of the virtual keyboard, including the step of running 
processes on the computer to capture gesture images, to classify each type of gesture image into 
a respective one of a plurality of classes depending on the type of gesture, and to associate each 
of the classes with one or more possible keys of the invisible keyboard, assign a probability to 
each of said possible keys, and integrate the probabilities assigned to the possible keys to identify 
a word for a sequence of gestures, and wherein the probability is computed using HMM. 

Claim 10: A method of typing using a virtual keyboard having a multitude of virtual keys, 
comprising the steps: 

making typing gestures relative to a virtual, invisible keyboard made without 
touching any touch sensors and without any real keyboard; 

sensing the typing gestures; and 

producing, from the sensed typing gestures, gesture associated textual output 
including the step of running processes on a computer to capture gesture images, classify each 
type of gesture image into a respective one of a plurality of classes depending on the type of 
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gesture, and to associate each of the classes with one or more possible keys of the invisible 
keyboard, assign a probability to each of said possible keys, and integrate the probabilities 
assigned to the possible keys to identify a word for a sequence of gestures. 

Claim 11: A method according to Claim 10, wherein the typing gestures are made by a person, 
and further comprising the steps of providing feedback to the person on texture output associated 
with the gestures. 

Claim 12: A method according to Claim 11, wherein the step of providing feedback includes the 
step of displaying an image of typing keys associated with the gestures. 

Claim 14: A method of typing using a virtual invisible keyboard, comprising the steps: 

making typing gestures relative to a virtual, invisible keyboard made without touching any touch 

sensors and without any real keyboard; 

sensing the typing gestures; and 

producing, from the sensed typing gestures, gesture associated textual output: and 
wherein the producing step includes the steps of classifying each type of gesture into a respective 
one of a plurality of classes depending on the type of gesture, associating each of said classes 
with one or more possible keys of the invisible keyboard, assigning a probability to each of said 
possible keys, and integrating the probabilities assigned to the possible keys to identifying a 
word for a sequence of gestures. 

Claim 15: A method according to Claim 14, wherein the producing step further includes the step 
of associating gesture classes with individual typing keys. 

Claim 16: A method according to Claim 14, further comprising providing training data in words 
or sentences with certain timing data. 

Claim 17: A typing system using a virtual, invisible keyboard, comprising 

means for sensing typing gestures made relative to a virtual, invisible keyboard made without 

touching any touch sensors and without any real keyboard; and 
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means for producing, from the sensed typing gestures, gesture associated textual 
output said producing means including a computer and processes running on the computer to 
capture gesture images, to classify each type of gesture image into a respective one of a plurality 
of classes depending on the type of gesture, and associate each of the classes with one or more 
possible keys of the invisible keyboard, assign a probability to each of said possible keys, and 
integrate the probabilities assigned to the possible keys to identify a word for a sequence of 
gestures. 

Claim 18: A system according to Claim 17, wherein the typing gestures are made by a person, 
and further comprising means for providing feedback to the person on texture output associated 
with the gestures. 

Claim 19: A system according to Claim 18, wherein the means for providing feedback includes 
means for displaying an image of typing keys associated with the gestures. 

Claim 21 : A typing system using a virtual invisible keyboard, comprising 

means for sensing typing gestures made relative to a virtual, invisible keyboard 

made without touching any touch sensors and without any real keyboard; and 

means for producing, from the sensed typing gestures, gesture associated textual 

output; and 

wherein the producing means includes means for classifying each type of gesture 
into a respective one of a plurality of classes depending on the type of gesture, associating each 
of said classes with one or more possible keys of the invisible keyboard, assigning a probability 
to each of said possible keys, and integrating the probabilities assigned to the possible keys to 
identifying a word for a response of gestures. 

Claim 22: A system according to Claim 21, wherein the producing means further includes 
means for associating gesture classes with individual typing keys. 
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Claim 23: A method according to Claim 10, wherein the step of running processes on the 
computer includes the step of mapping the sensed typing gestures to keys of the keyboard based 
on a statistical machine that interprets sequences of typing like gesture classes as intended words 
based on user typing gesture models. 

Claim 24: A method according to Claim 10, wherein gesture input is represented as wave forms 
that are digitized and clustered in gesture frames, and then processed by HMM machines that 
represent difference models for different typing patterns; and the step of running processes 
includes the step of using two statistical components, a first statistical component to map 
gestures to keys, and a second, language model component to map keys to letters and words. 
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